Electronic apparatus, method for controlling electronic apparatus, and control program

ABSTRACT

To divide an image capture region into multiple regions for which different image capture conditions are set and to generate multiple moving images corresponding to the multiple regions. An electronic apparatus includes an image sensor that captures first and second moving images in first and second regions of an image capture region on different image capture conditions, the second region differing from the first region, and a moving image generation unit that generates the first and second moving images captured in the first and second regions.

This application is a continuation of U.S. patent application Ser. No.16/986,644 filed on Aug. 6, 2020, which in turn is a continuation ofU.S. patent application Ser. No. 16/395,351 filed on Apr. 26, 2019,which in turn is a continuation of U.S. patent application Ser. No.14/910,428 filed on Feb. 5, 2016, which in turn is a National Stage ofInternational Patent Application PCT/JP2014/070874 filed Aug. 7, 2014,which claims the benefit of Japanese Patent Application No. 2013-167308filed on Aug. 12, 2013. The disclosure of each of the prior applicationsis incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present invention relates to an electronic apparatus, a method forcontrolling an electronic apparatus, and a control program.

BACKGROUND ART

Electronic apparatuses each including an image sensor in which aback-illuminated image-capture chip and a signal processing chip arestacked (hereafter referred to as a stacked image sensor) have beenproposed (for example, see Patent Literature 1). In a stacked imagesensor, a back-illuminated image capture chip and a signal processingchip are stacked so as to be connected via micro-bumps corresponding toblocks each including multiple pixels.

CITATION LIST Patent Literature

-   [Patent Literature 1] Japanese Unexamined Patent Application    Publication No. 2006-49361

SUMMARY OF INVENTION Technical Problem

However, there have been proposed only a few electronic apparatusesincluding a stacked image sensor that captures images on amultiple-block basis. Accordingly, the usability of electronicapparatuses including a stacked image sensor has not been sufficientlyimproved.

An object of an aspect of the present invention is to divide an imagecapture region into multiple regions for which different image captureconditions are set and to generate multiple moving images correspondingto the multiple regions.

Solution to Problem

A first aspect of the present invention provides an electronic apparatusincluding an image sensor configured to capture a first moving image anda second moving image in a first region and a second region,respectively, of an image capture region on different image captureconditions, the second region differing from the first region, and amoving image generation unit configured to generate the first movingimage and the second moving image captured in the first region and thesecond region.

A second aspect of the present invention provides an electronicapparatus including an image sensor configured to generate a firstmoving image and a second moving image in a first region and a secondregion, respectively, of an image capture region, the first regiondiffering from the second region.

A third aspect of the present invention provides an electronic apparatusincluding a setting unit configured to set image capture conditions forfirst and second regions of an image capture region of an image sensor,the second region differing from the first region and a moving imagegeneration unit configured to generate a first moving image and a secondmoving image captured in the first region and the second region,respectively.

A fourth aspect of the present invention provides an electronicapparatus including an image capture unit including an image sensor, adivision unit configured to divide an image capture region of the imagesensor into at least first and second regions, an image capture controlunit configured to set different image capture conditions for the firstand second regions, and a moving image generation unit configured togenerate a first moving image from images captured in the first regionand to generate a second moving image from images captured in the secondregion.

A fifth aspect of the present invention provides a method forcontrolling an electronic apparatus including an image capture unit, themethod including dividing an image capture region of the image sensorinto at least first and second regions, setting different image captureconditions for the first and second regions, and generating a firstmoving image from an image captured in the first region and generating asecond moving image from an image captured in the second region.

A sixth aspect of the present invention provides a control program forcausing a control unit of an electrode apparatus including an imagecapture unit to perform a dividing process of dividing an image captureregion of the image sensor into at least first and second regions, animage capture control process of setting different image captureconditions for the first and second regions, and a moving imagegeneration process of generating a first moving image from an imagecaptured in the first region and generating a second moving image froman image captured in the second region.

Advantageous Effects of the Invention

According to the aspects of the present invention, it is possible togenerate multiple moving images corresponding to multiple regions forwhich different image capture conditions are set.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a sectional view of a stacked image sensor.

FIG. 2 is a diagram showing the pixel array of an image capture chip anda unit group.

FIG. 3 is a circuit diagram of a unit group of the image capture chip.

FIG. 4 is a block diagram showing the functional configuration of theimage sensor.

FIG. 5 is a block diagram showing the configuration of an electronicapparatus according to a first embodiment.

FIG. 6 is a diagram showing an example of the display screen of adisplay unit.

FIG. 7 is a function block diagram of an image processing unit and asystem control unit.

FIG. 8 includes diagrams showing block arrangement patterns.

FIG. 9 is a flowchart showing an image capture operation performed bythe system control unit.

FIG. 10 is a flowchart showing a block arrangement pattern settingprocess.

FIG. 11 is a diagram showing an example of the second arrangementpattern set in a second moving image mode.

FIG. 12 is a diagram showing an example display of a live view image inan image display region.

FIG. 13 is a diagram showing the range and position of an electroniczoom in the image display region.

FIG. 14 is a diagram showing the range and position of a sub-screen inthe image display region.

FIG. 15 is a flowchart showing operations in the second moving imagemode.

FIG. 16 is a timing chart showing charge accumulation timings in thesecond moving image mode.

FIG. 17 is a block diagram showing the configuration of an image capturedevice and an electronic apparatus according to a second embodiment.

EMBODIMENTS OF THE INVENTION

Hereafter, embodiments of the present invention will be described withreference to the drawings. However, the present invention is not limitedthereto. To clarify the embodiments, the drawings are scaled asappropriate, for example, partially enlarged or highlighted.

First Embodiment

FIG. 1 is a sectional view of a stacked image sensor. A stacked imagesensor 100 is disclosed in Japanese Patent Application No. 2012-139026previously filed by the present applicant. The image sensor 100 includesan image-capture chip 113 configured to output a pixel signalcorresponding to incident light, a signal processing chip 111 configuredto process the pixel signal, and a memory chip 112 configured to storethe pixel signal. The image-capture chip 113, signal processing chip111, and memory chip 112 are stacked and electrically connected to oneanother via conductive bumps 109 such as Cu.

As shown in FIG. 1, incident light enters the image sensor 100 in apositive z-axis direction mainly shown by an outline arrow. In thepresent embodiment, the incident light entry surface of theimage-capture chip 113 is referred to as a back surface. Further, asshown by coordinate axes, the direction which is perpendicular to thez-axis and oriented to the left side of the drawing is referred to as apositive x-axis direction, and the direction which is perpendicular tothe z- and x-axes and oriented to the viewer is referred to as apositive y-axis direction. In the following some drawings, coordinateaxes are shown using the coordinate axes of FIG. 1 as a reference sothat the orientations of such drawings are understood.

One example of the image-capture chip 113 is a back-illuminated MOSimage sensor. A PD layer 106 is disposed on the back surface of a wiringlayer 108. The PD layer 106 includes multiple photodiodes (PDs) 104disposed two-dimensionally and configured to accumulate chargecorresponding to incident light and transistors 105 disposed in a mannercorresponding to the PDs 104.

Color filters 102 are disposed over the incident light entry surface ofthe PD layer 106 with a passivation film 103 therebetween. The colorfilters 102 are each a filter which transmits a particular wavelengthrange of visible light. That is, the color filters 102 include multiplecolor filters which transmit different wavelength ranges and arearranged in a particular manner so as to correspond to the PDs 104. Thearrangement of the color filters 102 will be described later. A set of acolor filter 102, a PD 104, and a transistor 105 forms one pixel.

Microlenses 101 are disposed on the incident light entry sides of thecolor filters 102 in a manner corresponding to the pixels. Themicrolenses 101 condense incident light toward the corresponding PDs104.

The wiring layer 108 includes lines 107 configured to transmit pixelsignals from the PD layer 106 to the signal processing chip 111. Thelines 107 may be multilayered and may include passive and activeelements. Multiple bumps 109 are disposed on the front surface of thewiring layer 108 and aligned with multiple bumps 109 disposed on theopposite surface of the signal processing chip 111. The aligned bumps109 are bonded together and electrically connected together, forexample, by pressurizing the image-capture chip 113 and signalprocessing chip 111.

Similarly, multiple bumps 109 are disposed on the opposite surfaces ofthe signal processing chip 111 and memory chip 112 and aligned with eachother. The aligned bumps 109 are bonded together and electricallyconnected together, for example, by pressurizing the signal processingchip 111 and memory chip 112.

The methods for bonding the bumps 109 together include Cu bump bondingusing solid phase diffusion, as well as micro-bump bonding using soldermelting. For the bumps 109, it is only necessary to provide, forexample, one bump or so with respect to one unit group (to be discussedlater). Accordingly, the size of the bumps 109 may be larger than thepitch between the PDs 104. Further, bumps which are larger than thebumps 109 corresponding to a pixel region having the pixels arrangedtherein (a pixel region 113A shown in FIG. 2) may be additionallyprovided in peripheral regions other than the pixel region.

The signal processing chip 111 includes a through-silicon via (TSV) 110configured to connect together circuits disposed on the front and backsurfaces thereof. The TSV 110 is disposed in a peripheral region.Alternatively, the TSV 110 may be disposed in a peripheral region of theimage-capture chip 113 or in the memory chip 112.

FIG. 2 is a diagram showing the pixel array of the image-capture chipand a unit group. In FIG. 2, the image-capture chip 113 is observed fromthe back side. The pixel region 113A is the pixel-arranged region of theimage-capture chip 113. In the pixel region 113A, 20 million or morepixels are arranged in a matrix. In an example shown in FIG. 2, fouradjacent pixels×four adjacent pixels, that is, 16 pixels form one unitgroup 131. Grid lines in FIG. 2 show a concept that adjacent pixels aregrouped into unit groups 131. The number of pixels forming the unitgroups 131 is not limited to that described above and may be on theorder of 1000, for example, 32 pixels×64 pixels, or may be 1000 or moreor less than 1000.

As shown in a partial enlarged view of the pixel region 113A, one unitgroup 131 includes four so-called Bayer arrays which each includes fourpixels, that is, green pixels Gb, Gr, a blue pixel B, and a red pixel Rand which are arranged vertically and horizontally. The green pixels areeach a pixel having a green filter as a color filter 102 and receivelight in the green wavelength band of incident light. Similarly, theblue pixel is a pixel having a blue filter as a color filter 102 andreceives light in the blue wavelength band. The red pixel is a pixelhaving a red filter as a color filter 102 and receives light in the redwavelength band.

FIG. 3 is a circuit diagram of a unit group of the image-capture chip.In FIG. 3, a rectangle surrounded by a dotted line as a representativeshows the circuit of one pixel. At least part of each transistordescribed below corresponds to one transistor 105 in FIG. 1.

As described above, one unit group 131 includes 16 pixels. Sixteen PDs104 included in these pixels are connected to corresponding transfertransistors 302. The gates of the transfer transistors 302 are connectedto a TX line 307 through which a transfer pulse is supplied. In thepresent embodiment, the TX line 307 is shared by the 16 transfertransistors 302.

The drain of each transfer transistor 302 is connected to the source ofa corresponding reset transistor 303, and so-called floating diffusionFD (charge detection unit) therebetween is connected to the gate of acorresponding amplifier transistor 304. The drains of the resettransistors 303 are connected to a Vdd line 310 through which apower-supply voltage is supplied. The gates of the reset transistors 303are connected to a reset line 306 through which a reset pulse issupplied. In the present embodiment, the reset line 306 is shared by the16 reset transistors 303.

The drains of the amplifier transistors 304 are connected to the Vddline 310, through which a power-supply voltage is supplied. The sourcesof the amplifier transistors 304 are connected to the drains ofcorresponding select transistors 305. The gates of the selecttransistors 305 are connected to corresponding decoder lines 308 throughwhich a selection pulse is supplied. In the present embodiment, thedifferent decoder lines 308 are disposed with respect to the 16 selecttransistors 305. The sources of the select transistors 305 are connectedto a shared output line 309. A load current source 311 supplies acurrent to the output line 309. That is, the output line 309 withrespect to the select transistors 305 is formed by a source follower.The load current source 311 may be disposed in any of the image-capturechip 113 and signal processing chip 111.

Described below is the flow from when the accumulation of charge startsto when pixel signals are outputted after the accumulation ends. Resetpulses are applied to the reset transistors 303 through the reset line306. Simultaneously, transfer pulses are applied to the transfertransistors 302 through the TX line 307. Thus, the potentials of the PDs104 and floating diffusion FD are reset.

When the application of the transfer pulses is released, the PDs 104convert received incident light into charge and accumulate it.Subsequently, when transfer pulses are applied again with reset pulsesnot being applied, the charge accumulated in each PD 104 is transferredto the corresponding floating diffusion FD. Thus, the potential of thefloating diffusion FD is changed from the reset potential to the signalpotential after the charge accumulation. When selection pulses areapplied to the select transistors 305 through the decoder lines 308, thevariation in the signal potential of each floating diffusion FD istransmitted to the output line 309 through the corresponding amplifiertransistor 304 and select transistor 305. Based on such a circuitoperation, the unit pixels output, to the output line 309, pixel signalscorresponding to the reset potentials and pixel signals corresponding tothe signal potentials.

As shown in FIG. 3, in the present embodiment, the reset line 306 and TXline 307 are shared by the 16 pixels forming the unit group 131. Thatis, reset pulses and transfer pulses are simultaneously applied to allthe 16 pixels. Accordingly, all the pixels forming the unit group 131start to accumulate charge at the same timing and end the chargeaccumulation at the same timing. Note that selection pulses aresequentially applied to the select transistors 305 and therefore pixelsignals corresponding to the accumulated charge are selectivelyoutputted to the output line 309. Different reset lines 306, TX lines307, and output lines 309 are disposed for the respective unit groups131.

By constructing the circuit on the basis of unit groups 131 as describedabove, the charge accumulation time can be controlled for each unitgroup 131. In other words, it is possible to cause the unit groups 131to output pixel signals based on different charge accumulation times.More specifically, by causing another unit group 131 to accumulatecharge several times and to output pixel signals each time while oneunit group 131 is caused to accumulate charge once, it is possible tocause the unit groups 131 to output moving image frames at differentframe rates.

FIG. 4 is a block diagram showing the functional configuration of theimage sensor. An analog multiplexer 411 sequentially selects 16 PDs 104forming one unit group 131 and causes each selected PD 104 to output apixel signal to an output line 309 disposed in a manner corresponding tothe unit group 131. The multiplexer 411 is formed along with the PDs 104in the image-capture chip 113.

The analog pixel signals outputted through the multiplexer 411 areamplified by an amplifier 412 which is formed in the signal processingchip 111. The pixel signals amplified by the amplifier 412 are subjectedto correlated double sampling (CDS) and analog-to-digital (A/D)conversion by a signal processing circuit 413 formed in the signalprocessing chip 111 and configured to perform CDS and A/D conversion.Since the pixel signals are subjected to CDS by the signal processingcircuit 413, the noise in the pixel signals is reduced. TheA/D-converted pixel signals are passed to a demultiplexer 414 and thenstored in corresponding pixel memories 415. The demultiplexer 414 andpixel memories 415 are formed in the memory chip 112.

An arithmetic circuit 416 processes the pixel signals stored in thepixel memories 415 and passes the resulting signals to a subsequentimage processing unit. The arithmetic circuit 416 may be disposed in anyof the signal processing chip 111 and memory chip 112. While theelements connected to the single unit group 131 are shown in FIG. 4,these elements are disposed for each unit group 131 in practice andoperate in parallel. Note that the arithmetic circuit 416 need notnecessarily be disposed for each unit group 131. For example, a singlearithmetic circuit 416 may sequentially refer to and process the valuesin the pixel memories 415 corresponding to the respective unit groups131.

As described above, the output lines 309 are disposed in a mannercorresponding to the respective unit groups 131. In the image sensor100, the image-capture chip 113, signal processing chip 111, and memorychip 112 are stacked. Accordingly, by using, as the output lines 309,the bumps 109 electrically connecting between the chips, the lines canbe routed without enlarging the chips in the surface direction.

Next, blocks set in the pixel region 113A (see FIG. 2) of the imagesensor 100 will be described. In the present embodiment, the pixelregion 113A of the image sensor 100 is divided into multiple blocks.Each block includes at least one unit group 131. Pixels included in therespective blocks are controlled by different control parameters. Thatis, the control parameter varies between pixel signals acquired frompixels included in one block and pixel signals acquired from pixelsincluded in another block. Examples of a control parameter include thecharge accumulation time or frequency, the frame rate, the gain, thethinning-out rate, the number of rows or columns whose pixel signals aresummed up, and the digitized bit number. The control parameters may beparameters used in image processing following the acquisition of imagesignals from the pixels.

As used herein, the charge accumulation time refers to the time fromwhen the PDs 104 start to accumulate charge to when they end theaccumulation. The charge accumulation frequency refers to the frequencywith which the PDs 104 accumulate charge per unit time. The frame raterefers to the number of frames processed (displayed or recorded) perunit time in a moving image. The frame rate is expressed in frames persecond (fps). As the frame rate is increased, a subject (i.e., subjectswhose images are to be captured) moves more smoothly in a moving image.

The gain refers to the gain factor (amplification factor) of theamplifier 412. By changing the gain, the ISO sensitivity can be changed.The ISO sensitivity is a standard for photographic films developed bythe ISO and represents the level of the weakest light which aphotographic film can record. Typically, the sensitivity of imagesensors is represented by the ISO sensitivity. In this case, the abilityof the image sensor 100 to capture light is represented by the value ofthe ISO sensitivity. When the gain is increased, the ISO sensitivity isincreased as well. For example, when the gain is doubled, the electricalsignal (pixel signal) is doubled as well. Thus, appropriate brightnessis obtained even when the amount of incident light is halved. However,the increase in gain amplifies noise included in the electric signal,thereby increasing noise.

The thinning-out rate refers to the ratio of the number of pixels fromwhich pixel signals are not read to the total number of pixels in apredetermined region. For example, a thinning-out rate of apredetermined region of 0 means that pixel signals are read from allpixels in the predetermined region. A thinning-out rate of apredetermined region of 0.5 means that pixel signals are read from halfthe pixels in the predetermined region. Specifically, where a unit group131 is a Bayer array, one Bayer array unit from which pixel signals areread and one Bayer array unit from which pixel signals are not read arealternately set in the vertical direction, that is, two pixels (tworows) from which pixel signals are read and two pixels (two rows) fromwhich pixel signals are not read are alternately set in the verticaldirection. On the other hand, when the pixels from which pixel signalsare read are thinned out, the resolution of images is reduced. However,20 million or more pixels are arranged in the image sensor 100 andtherefore, even when the pixels are thinned out, for example, at athinning-out rate of 0.5, images can be displayed with 10 million ormore pixels. For this reason, the user (operator) seems not to worryabout such a resolution reduction.

The number of rows whose pixel signals are summed up refers to thenumber of vertically adjacent pixels whose pixel signals are summed up.The number of columns whose pixel signals are summed up refers to thenumber of horizontally adjacent pixels whose pixel signals are summedup. Such a summation process is performed, for example, in thearithmetic circuit 416. When the arithmetic circuit 416 sums up pixelsignals of a predetermined number of vertically or horizontally adjacentpixels, there is obtained an effect similar to that obtained by thinningout the pixels at a predetermined thinning-out rate and reading pixelsignals from the resulting pixels. In the summation process, an averagevalue may be calculated by dividing the sum of the pixel signals by therow number or column number obtained by the arithmetic circuit 416.

The digitized bit number refers to the number of bits of a digitalsignal converted from an analog signal by the signal processing circuit413. As the number of bits of a digital signal is increased, luminance,color change, or the like is represented in more detail.

In the present embodiment, the accumulation conditions refer to theconditions on the accumulation of charge in the image sensor 100.Specifically, the accumulation conditions refer to the chargeaccumulation time or frequency, frame rate, and gain of the controlparameters. Since the frame rate can change according to the chargeaccumulation time or frequency, it is included in the accumulationconditions. Similarly, the correct amount of exposure can changeaccording to the gain, and the charge accumulation time or frequency canchange according to the correct amount of exposure. Accordingly, thegain is included in the accumulation conditions.

The image-capture conditions refer to conditions on image-capture of asubject. Specifically, the image-capture conditions refer to controlparameters including the accumulation conditions. The image-captureconditions includes control parameters for controlling the image sensor100 (e.g., the charge accumulation time or frequency, frame rate, gain),as well as control parameters for controlling reading of signals fromthe image sensor 100 (e.g., thinning-out rate), and control parametersfor processing signals from the image sensor 100 (e.g., the number ofrows or columns whose pixel signals are summed up, digitized bit number,and control parameters used when an image processing unit 30 (to bediscussed later) processes images).

FIG. 5 is a block diagram showing the configuration of an electronicapparatus according to the first embodiment. The electronic apparatus 1shown in FIG. 5 includes digital cameras, smartphones, mobile phones,and personal computers which each have an image capture function. Asshown in FIG. 5, an electronic apparatus 1 includes a lens unit 10, animage-capture unit 20, the image processing unit 30, a work memory 40, adisplay unit 50, an operation unit 55, a recording unit 60, and a systemcontrol unit 70. The lens unit 10 is an image-capture optical systemincluding multiple lenses. The lens unit 10 guides a pencil of rays froma subject to the image-capture unit 20. The lens unit 10 may be integralwith the electronic apparatus 1 or may be an interchangeable lens whichis detachable from the electronic apparatus 1. The lens unit 10 may alsoinclude a focus lens or zoom lens.

The image-capture unit 20 includes the image sensor 100 and a drive unit21. The drive unit 21 is a control circuit configured to control thedrive of the image sensor 100 in accordance with an instruction from thesystem control unit 70. Specifically, the drive unit 21 controls thecharge accumulation time or frequency, which is a control parameter, bycontrolling the timing (or the cycle of the timing) when reset pulses ortransfer pulses are applied to the reset transistors 303 or transfertransistors 302, respectively. The drive unit 21 also controls the framerate by controlling the timing (or the cycle of timing) when resetpulses, transfer pulses, or selection pulses are applied to the resettransistors 303, transfer transistor 302, or select transistors 305,respectively. The drive unit 21 also controls the thinning-out rate bysetting pixels to which reset pulses, transfer pulses, and selectionpulses are applied.

The drive unit 21 also controls the ISO sensitivity of the image sensor100 by controlling the gain (also called the gain factor oramplification factor) of the amplifier 412. The drive unit 21 also setsthe number of rows or columns whose pixel signals are summed up bytransmitting an instruction to the arithmetic circuit 416. The driveunit 21 also sets the digitized bit number by transmitting aninstruction to the signal processing circuit 413. The drive unit 21 alsosets blocks in the pixel region (image-capture region) 113A of the imagesensor 100. As seen above, the drive unit 21 serves as an image sensorcontrol unit that causes the image sensor 100 to capture an image underimage-capture conditions which vary among the blocks and then to outputpixel signals. The system control unit 70 transmits an instruction aboutthe position, shape, range, or the like of blocks to the drive unit 21.

The image sensor 100 passes the pixel signals from the image sensor 100to the image processing unit 30. The image processing unit 30 generatesimage data by performing various types of image processing on raw datacomposed of the pixel signals of the pixels using the work memory 40 aswork space. The image processing unit 30 includes a first imageprocessing unit 30A and a second image processing unit 30B. When theload of image processing is high, the processing is distributed to thefirst image processing unit 30A and second image processing unit 30B.The first image processing unit 30A and second image processing unit 30Bthen perform the distributed processing in parallel.

In the present embodiment, as will be described later, the systemcontrol unit 70 (specifically, a division unit 71 shown in FIG. 7)divides the pixel region (image capture region) 113A of the image sensor100 into at least first and second regions. The system control unit 70(specifically, an image capture control unit 72 shown in FIG. 7) alsocontrols the drive of the image sensor 100 so that the image sensor 100captures images in the first and second regions on different imagecapture conditions. In this case, for example, the first imageprocessing unit 30A performs image processing on signals from the firstregion, and the second image processing unit 30B performs imageprocessing on signals from the second region. Note that the pixel region(image capture region) 113A of the image sensor 100 need not be dividedinto the two regions composed of the first and second regions and may bedivided into multiple regions composed of a first region, a secondregion, a third region, and the like. In this case, image processingwith respect to the multiple regions is distributed to the first imageprocessing unit 30A and second image processing unit 30B as appropriate.The distribution of image processing may be previously determined on thebasis of the number of regions obtained by division, the ranges of theregions, or the like. The system control unit 70 may determine thedistribution on the basis of the number of regions obtained by division,the ranges of the regions, or the like.

The image processing unit 30 performs various types of image processing.For example, the image processing unit 30 performs color signalprocessing (tone correction) on signals obtained from a Bayer array soas to generate RGB image signals. The image processing unit 30 thenperforms image processing such as white balance adjustment, sharpnessadjustment, gamma correction, gradation adjustment, or the like on theRGB image signals. The image processing unit 30 compresses the resultingsignals in a predetermined compression format (JPEG format, MPEG format,or the like), if necessary. The image processing unit 30 then outputsthe resulting image data to the recording unit 60. The image processingunit 30 also outputs the image data to the display unit 50.

In the present embodiment, the image processing unit 30 performs theabove processes, as well as detects a main subject from the image data.As used herein, the term “main subject” refers to a subject which isnoted or assumed to be noted by the user (operator), of subjects whoseimages are to be captured. The number of main subjects in the image datais not limited to one, and multiple main subjects may be present (forexample, see FIG. 11).

Parameters referred to when the image processing unit 30 performs imageprocessing are also included in the control parameters (image captureconditions). For example, parameters such as color signal processing(tone correction), white balance adjustment, gradation adjustment, andcompressibility are included in the control parameters. The signals readfrom the image sensor 100 vary with the charge accumulation time or thelike, and the parameters referred to when image processing is performedalso vary with the variations in the signals. The image processing unit30 sets different control parameters for the respective blocks andperforms image processing such as color signal processing on the basisof the control parameters.

The image processing unit 30 extracts or discards frames correspondingto predetermined timings from multiple frames chronologically obtainedfrom the image capture unit 20. Thus, it is possible to reduce theamount of data to reduce the load on subsequent processes. The imageprocessing unit 30 also calculates one or more frames to be interpolatedbetween multiple frames chronologically obtained from the image captureunit 20 and then interpolates the calculated one or more frames betweenthe multiple frames. Thus, it is possible to play back moving images insuch a manner that the images move more smoothly. While the drive unit21 is configured to control the thinning-out rate, other configurationsmay be employed. For example, the image processing unit 30 or arithmeticcircuit 416 may control the thinning-out rate by discardingpredetermined pixel signals of pixel signals read from all the pixels bythe drive unit 21.

The work memory 40 temporarily stores image data or the like when theimage processing unit 30 processes images. The display unit 50 is, forexample, a liquid crystal display panel. As shown in FIG. 5, the displayunit 50 includes a display panel 51 and a touchscreen 52. The displayunit 51 displays images (still images, moving images, live view images)captured by the image capture unit 20, or various types of information.The touchscreen 52 is formed on the display screen of the display panel51. When the user touches the touchscreen 52 to perform an operationsuch as selection of a region in an image, the touchscreen 52 outputs asignal indicating the touched position to the system control unit 70.Also, when the user touches the touchscreen 52 to set the image capturemode, image capture condition, or screen change, the touchscreen 52outputs a signal indicating the touched position to the system controlunit 70.

The operation unit 55 includes a release switch, a moving image switch,and other types of operation switches operated by the user. Theoperation unit 55 outputs a signal corresponding to an operationperformed by the user to the system control unit 70. The user makes ashooting preparation such as automatic focusing (AF) or automaticexposure (AE) by pressing the release switch halfway.

The recording unit 60 has two card slots into which two recording media(first recording medium 61, second recording medium 62), such as memorycards, can be inserted. The recording unit 60 stores image datagenerated by the image processing unit 30 or various types of data inthe recording media (first recording medium 61, second recording medium62) inserted in the card slots. In the present embodiment, as describedabove, the first image processing unit 30A and second image processingunit 30B perform image processing on signals from the first region andsignals from the second region, respectively, in parallel. At this time,the first recording medium 61 stores image data based on the signalsfrom the first region in response to an operation of the release switchor moving image switch. Similarly, the second recording medium 62 storesimage data based on the signals from the second region in response to anoperation of the release switch or moving image switch. The recordingunit 60 also includes an internal memory. The recording unit 60 mayrecord, in the internal memory, the image data generated by the imageprocessing unit 30 or various types of data.

The system control unit 70 controls the entire processing and operationof the electronic apparatus 1. The system control unit 70 includes acentral processing unit (CPU) 70A. In the present embodiment, the systemcontrol unit 70 divides the image capture surface (pixel region 113A) ofthe image sensor 100 (image capture chip 113) into multiple blocks andcauses the image sensor 100 to capture images in the blocks withdifferent charge accumulation times (or charge accumulationfrequencies), different frame rates, and/or different gains. For thisreason, the system control unit 70 transmits the positions, shapes, andranges of the blocks and the accumulation conditions for the blocks tothe drive unit 21. The system control unit 70 also causes the imagesensor 100 to capture images in the blocks with different thinning-outrates, the different numbers of rows or columns whose pixel signals aresummed up, and/or different digitized bit numbers. For this reason, thesystem control unit 70 transmits, to the drive unit 21, the imagecapture conditions (thinning-out rates, the numbers of rows or columnswhose pixel signals are summed up, and digitized bit numbers) for theblocks. The image processing unit 30 performs image processing on imagecapture conditions (control parameters such as color signal processing,white balance adjustment, gradation adjustment, and compressibility)which vary among the blocks. For this reason, the image processing unit70 transmits, to the image processing unit 30, the image captureconditions (control parameters such as color signal processing, whitebalance adjustment, gradation adjustment, and compressibility) for theblocks.

The system control unit 70 records the image data generated by the imageprocessing unit 30 in the recording unit 60. The system control unit 70also outputs the image data generated by the image processing unit 30 tothe display unit 50 so that images are displayed on the display unit 50.The system control unit 70 also reads image data recorded in therecording unit 60 and outputs it to the display unit 50 so that imagesare displayed on the display unit 50. The images displayed on the firstdisplay unit 51 are still images, moving images, or live view images. Asused herein, the term “live view images” refer to images displayed onthe display unit 50 on the basis of image data sequentially generatedand outputted by the image processing unit 30. The user uses live viewimages to check images of the subject being captured by the imagecapture unit 20. Live view images are also called through images orpreview images.

FIG. 6 is a diagram showing an example of the display screen of thedisplay unit. As shown in FIG. 6, the display panel 51 of the displayunit 50 includes an image display region 510 and an operation buttondisplay region 520. The image display region 510 is a region fordisplaying images captured by the image capture unit 20, that is, stillimages, moving images, and live view images. The operation buttondisplay region 520 is a region for displaying menus on which the usersets image capture conditions or the like.

The operation button display region 520 is disposed near the imagedisplay region 510. Menus (menu images) for the user to set the imagecapture mode, image capture condition, and display screen are displayedin the operation button display region 520. An image capture mode button521 is a menu for the user to set the image capture mode. An ISOsensitivity button 522 and a frame rate button 523 are menus for theuser to set the image capture conditions. A screen change button 524 isa menu for the user to set the screen change. Hereafter, the imagecapture mode button 521, ISO sensitivity button 522, frame rate button523, and screen change button 524 will be simply referred to as the“image capture mode 521,” “ISO sensitivity 522,” “frame rate 523,” and“screen change 524,” respectively.

The image capture mode 521 is a button that the user presses to set(select) the image capture mode. The ISO sensitivity 522 is a buttonthat the user presses to set the ISO sensitivity (i.e., gain). The framerate 523 is a button that the user presses to set the frame rate ofmoving images. The screen change 524 is a button that the user pressesto display images captured in a partial region, over the entire displayscreen in an enlarged manner using an electronic zoom (digital zoom)(see FIGS. 13, 14) or to display an image on a sub-screen (see FIG. 14).

The image capture mode includes a still image mode in which still imagesare captured and a moving image mode in which moving images arecaptured. The moving image mode includes a first moving image mode and asecond moving image mode. The still image mode refers to an imagecapture mode in which the image sensor 100 captures still images of thesubject using the pixel region (image capture region) 113A thereof as asingle region without the division unit 71 dividing the pixel region113A. The still image mode is a typical still image capture mode.

The first moving image mode refers to an image capture mode in which theimage sensor 100 captures moving images of the subject using the pixelregion (image capture region) 113A thereof as a single region withoutthe division unit 71 dividing the pixel region 113A. The first movingimage mode is a typical moving image capture mode. The second movingimage mode is an image capture mode in which the division unit 71divides the pixel region 113A into multiple regions including at leastfirst and second regions and the image sensor 100 captures moving imagesof the same subject in each of the multiple regions.

The touchscreen 52 is disposed on the image display region 510 andoperation button display region 520. A touch region 510 a of thetouchscreen 52 is formed on the image display region 510. When the touchregion 510 a detects that it has been pressed (touched) by the user, itoutputs a detection signal indicating the pressed position to the systemcontrol unit 70.

A touch region 521 a is formed so as to overlap the image capture mode521. A touch region 522 a is formed so as to overlap the ISO sensitivity522. A touch region 523 a is formed so as to overlap the frame rate 523.A touch region 524 a is formed so as to overlap the screen change 524.When any of the touch regions 521 a to 524 a detects that it has beenpressed (touched) by the user, the touch region outputs a detectionsignal indicating the pressed position (the pressed touch region) to thesystem control unit 70.

The user may operate the operation unit 55 in place of the touchscreen52 to select the image capture mode, image capture condition, or displayscreen change.

FIG. 7 is a function block diagram of the image processing unit andsystem control unit shown in FIG. 5. As shown in FIG. 7, the first imageprocessing unit 30A includes an image generation unit (moving imagegeneration unit) 31A and a detection unit 32A. The image generation unit31A generates image data by performing various types of image processingon RAW data outputted from the image capture unit 20 and composed of thepixel signals from the pixels in the first region. The detection unit32A detects a main subject from the image data generated by the imagegeneration unit 31A. In the present embodiment, the detection unit 32Amakes a comparison among multiple pieces of image data chronologicallyobtained from live view images generated by the image generation unit31A and detects a moving subject as a main subject. The detection unit32A also detects a main subject using, for example, a face detectionfunction as described in Japanese Unexamined Patent ApplicationPublication No. 2010-16621 (US 2010/0002940). In addition to facedetection, the detection unit 32A also detects a human body included inthe image data as a main subject, as described in Japanese UnexaminedPatent Application Publication No. 2010-16621 (US 2010/0002940).

The second image processing unit 30B includes an image generation unit(moving image generation unit) 31B. The image generation unit 31Bgenerates image data by performing various types of image processing onRAW data outputted from the image capture unit 20 and composed of thepixel signals from the pixels in the second region. While the secondimage processing unit 30B does not include a detection unit, it mayinclude a detection unit. There may be employed a configuration in whichthe first image processing unit 30A does not include the detection unit32A and the second image processing unit 30B includes a detection unit.In the present embodiment, the image generation unit 31A and imagegeneration unit 31B may be collectively referred to as the imagegeneration unit 31.

The system control unit 70 includes a division unit 71, an image capturecontrol unit 72, a recording control unit 73, and a display control unit74. The division unit 71 divides the pixel region (image capture region)113A of the image sensor 100 into multiple regions on a block basis. Thedivision unit 71 divides the pixel region 113A into multiple regions onthe basis of a predetermined block arrangement pattern of the pixelregion 113A [see FIGS. 8(A) to 8(D)]. In response to the user touchingany menu for setting the image capture conditions (ISO sensitivity 522,frame rate 523), the image capture control unit 72 sets different imagecapture conditions for the multiple regions generated by the divisionunit 71. Also, in response to the user operating the release switch ormoving image switch, the image capture control unit 72 controls thedrive of the image sensor 100. Even during capture of live view images(that is, after starting an image capture operation following power-on),the drive control unit 72 controls the drive of the image sensor 100.

The recording control unit 73 records, in the recording unit 60, piecesof moving image data corresponding to the multiple regions generated bythe image generation unit 31. For example, the recording control unit 73records, in the first recording medium 61, image data corresponding tothe first region generated by the image generation unit 31A and records,in the second recording medium 62, image data corresponding to thesecond region generated by the image generation unit 31B. The displaycontrol unit 74 outputs the image data generated by the image generationunit 31 to the display unit 50 so that images (still images, movingimages, live view images) are displayed on the image display region 510of the display unit 50. Also, in response to the user operating thescreen change 524, the display control unit 74 changes the displayscreen. For example, in response to the user operating the screen change524, the display control unit 74 displays an image corresponding to apartial region, over the entire display screen in an enlarged mannerusing an electronic zoom and displays an image on the sub-screen.

The division unit 71, image capture control unit 72, recording controlunit 73, and display control unit 74 of the system control unit 70 areimplemented when the CPU 70A performs processing on the basis of acontrol program.

Next, block arrangement patterns set by the division unit 71 will bedescribed. FIG. 8 includes diagrams each showing a block arrangementpattern, in which FIG. 8(A) shows a first block arrangement pattern;FIG. 8(B) shows a second block arrangement pattern; FIG. 8(C) shows athird block arrangement pattern; and FIG. 8(D) shows a fourth blockarrangement pattern.

The first block arrangement pattern shown in FIG. 8(A) is a blockarrangement pattern in which the pixel region 113A is divided into tworegions, first and second regions. In the first block arrangementpattern, the first region of the pixel region 113A is composed of blocksin (2m−1)th columns, and the second region thereof is composed of blocksin (2m)th columns. That is, the blocks in the pixel region 113A aregrouped into the odd columns and even columns. As used herein, m is apositive integer (m=1, 2, 3, etc.).

The second block arrangement pattern shown in FIG. 8(B) is also a blockarrangement pattern in which the pixel region 113A is divided into tworegions, first and second regions. In the second block arrangementpattern, the first region of the pixel region 113A is composed of blocksin (2n−1)th rows, and the second region thereof is composed of blocks in(2n)th rows. That is, the blocks in the pixel region 113A are groupedinto the odd rows and even rows. As used herein, n is a positive integer(n=1, 2, 3, etc.).

The third block arrangement pattern shown in FIG. 8(C) is also a blockarrangement pattern in which the pixel region 113A is divided into tworegions, first and second regions. In the third block arrangementpattern, the first region of the pixel region 113A is composed of blocksin (2m−1)th columns and in (2n−1)th rows and blocks in (2m)th columnsand in (2n)th rows. The second region thereof is composed of blocks in(2m)th columns and in (2n−1)th rows and blocks in (2m−1)th columns andin (2n)th rows. That is, the pixel region 113A is divided in such amanner that the blocks form a check pattern. As used herein, m and n arepositive integers (m=1, 2, 3, . . . ; n=1, 2, 3, . . . ).

The fourth block arrangement pattern shown in FIG. 8(D) is a blockarrangement pattern in which the pixel region 113A is divided into threeregions, first to third regions. In the fourth block arrangementpattern, the first region of the pixel region 113A is composed of blocksin (3m−2)th columns; the second region thereof is composed of blocks in(3m−1)th columns; and the third region thereof is composed of blocks in(3m)th columns. As used herein, m is a positive integer (m=1, 2, 3,etc.).

While, in FIG. 8, a small number of blocks are set in the pixel region113A to make it easy to see the block arrangement in each region, alarger number of blocks than the number of blocks shown in FIG. 8 may beset.

Next, an image capture operation according to the first embodiment willbe described. FIG. 9 is a flowchart showing an image capture operationperformed by the system control unit. FIG. 10 is a flowchart showing asetting process. In the process shown in FIG. 9, the system control unit70 starts to capture images when the electronic apparatus 1 is poweredon. Although not shown in FIG. 9, when the system control unit 70 startsto capture images, the display control unit 73 displays live view imagescaptured by the image capture unit 20 in the image display region 510 ofthe display unit 51, as well as displays the menus 521 to 524 in theoperation button display region 520 of the display panel 51. Since liveview images need not be images in which the subject moves smoothly, thedrive control unit 72 controls the drive of the image sensor 100 so thatthe image sensor 100 captures images at a low frame rate.

The user selects the image capture mode by touching the image capturemode 521. The division unit 71 identifies the image capture mode thatthe user has selected by touching the image capture mode 521 (step S1).The division unit 71 determines whether the image capture mode selectedby the user is the still image mode (step S2). If the division unit 71determines that the image capture mode is the still image mode, it setsthe image capture mode to the still image mode (step S3). Then, theimage capture control unit 72 determines whether the user has operatedthe release switch (whether the user has pressed it all the wayfollowing halfway) (step S4).

If the image capture control unit 72 determines that the user hasoperated the release switch, it causes the image capture unit 20 tocapture images in the still image mode (step S5). Also in step S5, therecording control unit 73 records still images captured by the imagecapture unit 20 in the recording unit 60 (first recording medium 61 orfirst recording medium 61). Also in step S5, the display control unit 74displays the still images captured by the image capture unit 20 in theimage display region 510 of the display panel 51. Image capture in thestill image mode is similar to typical still image capture and thereforewill not be described in detail.

If the division unit 71 determines in step S2 that the image capturemode is not the still image mode, that is, the image capture mode is themoving image mode, it determines whether the moving image mode is thefirst moving image mode (step S10). If the division unit 71 determinesthat the moving image mode is the first moving image mode, it sets theimage capture mode to the first moving image mode (step S11). Incontrast, if the division unit 71 determines that the moving image modeis not the first moving image mode, that is, the moving image mode isthe second moving image mode, it sets the image capture mode to thesecond moving image mode (step S12).

In step S11 or step S12, the division unit 71 performs the settingprocess shown in FIG. 10. In the process shown in FIG. 10, the divisionunit 71 instructs the image processing unit 30 (first image processingunit 30A) to detect main subjects (step S21). The detection unit 32Athen detects moving subjects and non-moving subjects by making acomparison among multiple pieces of image data chronologically obtainedfrom live view images. In the present embodiment, the detection unit 32Arecognizes a face in each piece of image data on the basis of the eyes,mouse, the color of the skin, and the like and detects the face as amain subject. In the present embodiment, the detection unit 32A detectsthe faces, as well as detects a human body (person) included in eachpiece of image data as a main subject. The detection unit 32A thenoutputs the detection result along with the pieces of image data to thesystem control unit 70. The division unit 71 checks whether there aremain subjects, on the basis of the detection result from the detectionunit 32A. The division unit 71 then sets a region(s) corresponding tothe main subject(s), if any, and the image capture mode in the pixelregion 113A (step S22).

Specifically, if the image capture mode is the first moving image mode,the division unit 71 does not divide the pixel region 113A into multipleregions. That is, the division unit 71 sets the entire pixel region 113Aas a single region. At this time, the division unit 71 outputs, to thedrive unit 21, an instruction signal instructing the drive unit 21 toset the entire pixel region 113A as a single region.

In contrast, if the image capture mode is the second moving image mode,the division unit 71 selects one of the block arrangement patterns shownin FIGS. 8(A) to 8(D). The division unit 71 then checks whether the mainsubject(s) is a moving subject, on the basis of the detection resultfrom the detection unit 32A. If the main subject(s) is not a movingsubject but a non-moving subject, the division unit 71 sets first andsecond regions in accordance with the third block arrangement patternshown in FIG. 8(C). If the main subject (s) is a moving subject, thedivision unit 71 identifies the moving direction(s) of the movingsubject(s). If the moving direction(s) of the moving subject(s) ismostly vertical, the division unit 71 sets first and second regions inaccordance with the first block arrangement pattern shown in FIG. 8(A).If the moving direction(s) of the moving subject (s) is mostlyhorizontal, the division unit 71 sets first and second regions inaccordance with the second block arrangement pattern shown in FIG. 8(B).Further, if the moving subject(s) is moving fast vertically, thedivision unit 71 sets first to third regions in accordance with thefourth block arrangement pattern shown in FIG. 8(D). In step S22, thedivision unit 71 outputs, to the drive unit 21, an instruction signalindicating the block positions and the like of the respective regions(first and second regions, first to third regions).

FIG. 11 is a diagram showing an example of the second block arrangementpattern set in the second moving image mode. Note that in FIG. 11, theblocks are scaled up in order to make it easy to see the blockarrangement. In practice, smaller blocks than the blocks shown in FIG.11 are set in the pixel region 113A. In the example shown in FIG. 11,the detection unit 32A detects persons O1, O2 who are playing soccer anda soccer ball O3, as main subjects (moving subjects). The division unit71 determines that the main subjects O1 to O3 are moving subjects andare moving mostly horizontally, on the basis of the detection resultfrom the detection unit 32A. As a result, the division unit 71 setsfirst and second regions in accordance with the second block arrangementpattern shown in FIG. 8(B). At this time, the division unit divides thepixel region 113A in such a manner that the main subjects O1 to O3 arecontained in both the first and second regions.

Referring back to FIG. 10, in response to the user touching the ISOsensitivity 522 or frame rate 523, the image capture control unit 72sets image capture conditions for the regions set in step S22 (the firstand second regions in the example shown in FIG. 11) (step S23).Specifically, the division unit 71 outputs, to the drive unit 21, asignal indicating the image capture conditions (gains, frame rates)selected by the user through the touch on the ISO sensitivity 522 orframe rate 523.

The image capture control unit 72 may be configured to, when the gain ischanged, automatically set a frame rate value such that optimum exposureis made. The image capture control unit 72 may also be configured to,when the frame rate is changed, automatically set a gain value such thatoptimum exposure is made. The image capture control unit 72 may also beconfigured to, when the gain or frame rate is changed, output aninstruction signal indicating a parameter (image capture condition),such as color signal processing, white balance adjustment, gradationadjustment, or compressibility, to the image processing unit 30 so thatoptimum image correction is performed.

If the user sets different frame rates for the first region and secondregion in step S23, the number of frames recorded per unit time variesbetween the first recording medium 61 and second recording medium 62.Accordingly, if the first recording medium 61 and second recordingmedium 62 have the same recording capacity, a difference occurs betweenthe recording time of a first moving image in the first recording medium61 and the recording time of a second moving image in the secondrecording medium 62. That is, if the frame rate of the first region ishigher than that of the second region, the recording time of the firstmoving image in the first recording medium 61 is shorter than that ofthe second moving image in the second recording medium 62.

Then, in response to the user touching the screen change 524, thedisplay control unit 74 sets (changes) the display screen (step S24).FIG. 12 is a diagram showing an example display of a live view image inthe image display region. FIG. 13 is a diagram showing the range andposition of the electronic zoom in the image display region. FIG. 14 isa diagram showing the range and position of the sub-screen in the imagedisplay region.

In FIG. 12, a live view image (an image in which multiple persons areplaying soccer) is being displayed in the image display region 510 withthe user not touching the screen change 524 in step S24. This live viewimage includes multiple persons, O1, O2, O4 to O7, a soccer ball O3, anda goal O8 as main subjects.

If the user selects the electronic zoom by touching the screen change524 in step S24, the display control unit 74 detects the main subjectsO1 to O8 on the basis of the detection result from the detection unit32A. The display control unit 74 also detects the main human subjectsO1, O2, O4 to O7 from the main subjects O1 to O8. Then, as shown in FIG.13, the display control unit 74 extracts a partial region 511 in whichmany of the main human subjects O1, O2, O4 to O7 are gathering, from theimage display region 510. Then, as shown in FIG. 14, the display controlunit 74 displays a live view image of the extracted partial region 511over the entire image display region 510 in an enlarged manner. At thistime, the image capture control unit 72 controls the drive unit 21 sothat the drive unit 21 does not capture images in the region (the regionwhose live view image is not being displayed) other than the partialregion 511 in the image display region 510. Thus, it is possible tosuppress power consumption when the live view image is being displayedusing the electronic zoom in an enlarged manner.

If the user selects the display of the sub-screen (particular region)512 by touching the screen change 524 in step S24, the display controlunit 74 displays the live view image captured in the first region orsecond region, on the sub-screen 512 shown in FIG. 14. Specifically,when the live view image captured in the first region is being displayedin the image display region 510, the display control unit 74 instructsthe image processing unit 30 to combine the live view images captured inthe first and second regions so that the live view image captured in thesecond region is displayed on the sub-screen 512. On the other hand,when the live view image captured in the second is being displayed inthe image display region 510, the display control unit 74 instructs theimage processing unit 30 to combine the live view images captured in thefirst and second regions so that the live view image captured in thefirst region is displayed on the sub-screen 512. The display controlunit 74 then outputs the live view image combined by the imageprocessing unit 30 to the display unit 50 so that the live view image isdisplayed in the image display region 510. In the example shown in FIG.14, the sub-screen 512 is disposed at the lower-right corner of theimage display region 510.

Referring back to FIG. 9, the image capture control unit 72 determineswhether the user has operated the moving image switch (step S13). If theimage capture control unit 72 determines that the user has operated themoving image switch, it causes the image capture unit 20 to captureimages in the moving image mode (first moving image mode or secondmoving image mode) (step S14). Image capture in the first moving imagemode is similar to typical moving image capture and therefore will notbe described in detail.

FIG. 15 is a flowchart showing operations in the second moving imagemode (step S14). In a process shown in FIG. 15, the division unit 71determines whether the user has set the block arrangement (i.e.,regions) (step S31). The image capture control unit 72 also determineswhether the user has set the image capture conditions (step S31). In thepresent embodiment, the user can set the block arrangement or imagecapture conditions prior to starting to capture moving images (see stepsS22, S23 in FIG. 10), as well as can set the block arrangement or imagecapture conditions after starting to capture moving images.

For example, the using sets the block arrangement by operating theoperation unit 55 and thus selecting one of the block arrangementpatterns of FIGS. 8(A) to 8(D) displayed on the display panel 51. Forexample, the using sets the image capture conditions (gain or framerate) by touching the ISO sensitivity 522 or frame rate 523, as in stepS23 in FIG. 10.

If the division unit 71 determines that the user has set the blockarrangement, it sets the block arrangement again (step S32).Specifically, the image capture control unit 72 outputs, to the driveunit 21, an instruction signal indicating the block positions and thelike of the respective regions (first and second regions, first to thirdregions). If the image capture control unit 72 determines that the userhas set the image capture condition, it sets the image captureconditions again (step S32). Specifically, the image capture controlunit 72 outputs an instruction signal indicating the image captureconditions (gain or frame rate) to the drive unit 21.

The display control unit 74 then determines whether the user hasselected the electronic zoom (that is, the user has touched the screenchange 524) (step S33). If the display control unit 74 determines thatthe user has selected the electronic zoom, it displays the partialregion 511 over the entire image display region 510 in an enlargedmanner, as described with reference to step S24 in FIG. 10 (step S34).The display control unit 74 also determines whether the user hasselected the display of the sub-screen 512 (that is, the user hastouched the screen change 524) (step S35). If the display control unit74 determines that the user has selected the display of the sub-screen512, it displays the sub-screen 512 at the lower-right corner of theimage display region 510, as described with reference to step S24 inFIG. 10 (step S36).

Then, the image capture control unit 72 controls the drive of the imagesensor 100 so that the image sensor 100 captures images of the subjectin the block arrangement set in step S22 or step S32; in the range ofthe electronic zoom (if the partial region 511 has been displayed in anenlarged manner using the electronic zoom in step S34); and on the imagecapture conditions set in step S23 or step S32 (step S37). The recordingcontrol unit 73 records, in the recording unit 60, image data of themoving images captured by the image sensor 100 and processed by theimage processing unit 30 (step S38). Specifically, the recording controlunit 73 records, in the first recording medium 61, image data of firstmoving images captured in the first region and records, in the secondrecording medium 62, image data of second moving images captured in thesecond region.

In step S38, the recording control unit 73 outputs, to the imageprocessing unit 30, an instruction signal instructing the imageprocessing unit 30 to interpolate the image data of the moving imagescaptured in the second region into the image data of the first movingimages captured in the first region. The recording control unit 73 alsooutputs, to the image processing unit 30, an instruction signalinstructing the image processing unit 30 to interpolate the image dataof the moving images captured in the first region into the image data ofthe second moving images captured in the second region. When the pixelregion (image capture region) 113A is divided into two regions (firstregion, second region), the resolution is reduced to half. On the otherhand, by performing the process described above, the image data ofdefect pixels which occur due to the division of the region isinterpolated. Thus, a reduction in resolution can be prevented.

Even when the display control unit 74 is displaying the moving imagescaptured in the partial region 511 of the first region over the entireimage display region 510 in an enlarged manner using the electronic zoom(step S34), the recording control unit 73, in step S38, records, in thesecond recording medium 62, the second moving images captured in thesecond region and records, in the first recording medium 61, the movingimages captured in the partial region 511 of the first region. Accordingto this configuration, it is possible to record not only the movingimages of the partial region 511 being displayed over the entire imagedisplay region 510 in an enlarged manner but also the moving imageswhich are not being displayed in the image display region 510.

When the display control unit 74 is displaying one of the moving imagecaptured in the partial region 511 of the first region and the secondmoving image over the entire image display region 510 and is displayingthe other moving image on the sub-screen 512 (step S37), the recordingcontrol unit 73, in step S38, records a moving image in which the onemoving image and the other moving image are combined (see FIG. 14), inthe recording unit 60. According to this configuration, it is possibleto record moving images captured at different angles of view (a movingimage captured in the range of the electronic zoom and a moving imagecaptured in the entire image display region 510), as a single movingimage. At this time, if the one moving image and the other moving imagehave different frame rates, the image processing unit 30 thins out theframes or adds frames so that the frame rates are equalized. Thus, it ispossible to record the moving images captured at different angles ofview as a single moving image in the first recording medium 61 or secondrecording medium 62.

FIG. 16 is a timing chart showing charge accumulation timings in thesecond moving image mode (image capture in step S37). In FIG. 16, forexample, first and second regions are set in the second moving imagemode. While the user operates the moving image switch in the secondmoving image mode, the image capture control unit 72 outputs, to thedrive unit 21, an instruction signal instructing the drive unit 21 tocapture moving images in the first region, as well as to capture movingimages in the second region. In FIG. 16, while the user operates themoving image switch, the drive unit 21 causes the image sensor 100 tocapture moving images by accumulating charge in the pixels of the firstregion for a charge accumulation time T1. Also, while the user operatesthe moving image switch, the drive unit 21 causes the image sensor 100to capture moving images by accumulating charge in the pixels of thesecond region for a charge accumulation time T2 which is longer than thecharge accumulation time T1. The frame rate varies with the chargeaccumulation time. Accordingly, the frame rate of moving images variesbetween when images are captured for the charge accumulation time T1 andwhen images are captured for the charge accumulation time T2. Forexample, the frame rate corresponding to the charge accumulation time T1of the first region is 60 fps, and the frame rate corresponding to thecharge accumulation time T2 of the second region is 30 fps. The chargeaccumulation time or frame rate is set in the image capture conditionsetting process in step S23 or step S32.

The pixel signals read from the pixels in the first region of the imagesensor 100 are amplified by the amplifier 412 using the gain indicatedby the image capture control unit 72 and then outputted to the imageprocessing unit 30. The image generation unit 31A identifies parametersused in image processing, such as color signal processing, on the basisof an instruction signal outputted from the image capture control unit72 and indicating the image capture conditions of the first region. Theimage generation unit 31A then generates first-region image data byperforming various types of image processing on RAW data composed of thepixel signals from the pixels in the first region on the basis of theparameters.

The pixel signals read from the pixels in the second region of the imagesensor 100 are amplified by the amplifier 412 using the gain indicatedby the image capture control unit 72 and then outputted to the imageprocessing unit 30. The image generation unit 31B identifies parametersused in image processing, such as color signal processing, on the basisof an instruction signal outputted from the image capture control unit72 and indicating the image capture conditions of the second region. Theimage generation unit 31B then generates second-region image data byperforming various types of image processing on RAW data composed of thepixel signals from the pixels in the second region on the basis of theparameters.

Since the frame rate of the first region is higher than that of thesecond region as described above, the main human subjects in the firstmoving images move more smoothly than the main human subjects in thesecond region.

As described above, the electronic apparatus 1 of the first embodimentincludes the image capture unit 20, which includes the image sensor 100,the division unit 71, which divides the pixel region 113A of the imagesensor 100 into at least first and second regions, the image capturecontrol unit 72, which sets different image capture conditions for thefirst and second regions, and the moving image generation unit 31, whichgenerates a first moving image from an image captured in the firstregion and generates a second moving image from an image captured in thesecond region. According to this configuration, it is possible togenerate multiple types of moving images corresponding to multipleregions for which different image capture conditions are set. Thus,multiple types of moving images can be generated in accordance with thesubject or image capture situation. That is, the electronic apparatus 1including the image sensor 100 provides improved usability.

The moving image generation unit generates a first moving image from animage of the same subject captured in the first region and generates asecond moving image from an image of the same subject captured in thesecond region. Thus, it is possible to generate multiple types of movingimages of the same subject. The division unit 71 equalizes the number ofthe pixels in the first region and the number of the pixels in thesecond region, that is, the division unit 71 uniformly disposes theblocks in the first region and second region of the image display region510. Thus, variations in the resolution in the image display region 510are prevented. The division unit 71 forms a first region from multiplediscrete regions (multiple discrete blocks). This prevents theresolution of moving images from decreasing locally. The division unit71 variably divides the pixel region into first and second regions.Thus, the pixel region can be divided into regions corresponding tovarious situations, such as the type of the subject.

The image capture control unit 72 sets at least different frame ratesfor the first region and second region of the image sensor 100, as imagecapture conditions. Thus, the user can obtain multiple types of movingimages captured on the different image capture conditions. The movingimage generation unit 31 corrects first and second moving images usingat least one of different white balances, different gradations, anddifferent tone corrections. Thus, the user can obtain multiple types ofmoving images processed on the basis of different parameters.

The moving image generation unit 31 interpolates the moving imagecaptured in the second region into the first moving image andinterpolates the moving image captured in the first region into thesecond moving image. According to this configuration, the image data ofdefect pixels which occur due to the division of the pixel region 113Ais interpolated, thereby preventing a reduction in resolution. Theelectronic apparatus 1 includes the recording control unit 73, whichrecords the first and second moving images generated by the moving imagegeneration unit 31 in the recording unit 60. Thus, the simultaneouslygenerated two types of moving images can be recorded.

The electronic apparatus 1 includes the display control unit 74, whichdisplays moving images generated by the image generation unit 31 on thedisplay unit 50. Thus, the user can check the moving images displayed onthe display unit 50. The display control unit 74 displays the movingimage captured in the partial region 511 of the first region on thedisplay unit 50 in an enlarged manner, and the recording control unit 73records the second moving image and the moving image captured in thepartial region 511, in the recording unit 60. According to thisconfiguration, it is possible to record not only the moving images ofthe partial region 511 being displayed over the entire region of thedisplay unit 50 (the entire image display region 510) in an enlargedmanner but also moving images in the entire image display region 510which are not being displayed.

The display control unit 74 displays one of the moving image captured inthe partial region 511 of the first region and the second moving imageon the display unit 50, as well as displays the other moving image inthe particular region (sub-screen 512) of the display unit 50. Therecording control unit 73 records a moving image in which the one movingimage and the other moving image are combined (see FIG. 14), in therecording unit 60. According to this configuration, it is possible torecord moving images captured at different angles of view, as a singlemoving image.

The detection unit 32A detects main subjects from moving imagesgenerated by the moving image generation unit 31, and the division unit71 divides the pixel region into first and second regions in such amanner that the main subjects are contained in both the first and secondregions. Thus, it is possible to generate a first moving image of themain subjects in the first region and to generate a second moving imageof the main subjects in the second region. The image sensor 100 has astructure in which a back-illuminated image capture chip and a signalprocessing chip are stacked. Thus, the volume required to contain theimage sensor 100 can be reduced. The drive of the image sensor 100 iscontrolled on the basis of an instruction from the system control unit70. Thus, the load of the system control unit 70 can be reduced, and theimage sensor 100 can be easily mounted on the electronic apparatus 1.

While the electronic apparatus 1 according to the first embodiment shownin FIG. 5 includes the display unit 50, the display unit 50 may bedisposed outside the electronic apparatus. In this case, the systemcontrol unit 70 and display unit 50 are each provided with acommunication unit that receives and transmits signals (image data,control signals, and the like) by wire or wirelessly. The imageprocessing unit 30 and system control unit 70 may be formed integrallywith each other. In this case, the respective functions of the imageprocessing unit 30 and system control unit 70 are implemented when asystem control unit including one or more CPUs performs processing onthe basis of a control program. While the image processing unit 30includes the two image processing units, 30A and 30B, it may includeonly one image processing unit.

Second Embodiment

A configuration of a second embodiment is obtained by dividing theelectronic apparatus 1 of the first embodiment into an image capturedevice 1A and an electronic apparatus 1B.

FIG. 17 is a block diagram showing the configuration of the imagingcapture device and electronic apparatus according to the thirdembodiment. In the configuration shown in FIG. 17, the image capturedevice 1A captures images of subjects. The image capture device 1Aincludes a lens unit 10, an image capture unit 20, an image processingunit 30, a work memory 40, an operation unit 55, a recording unit 60,and a first system control unit 75. The 10, image capture unit 20, imageprocessing unit 30, work memory 40, operation unit 55, and recordingunit 60 of the image capture device 1A are similar to those shown inFIG. 5. Accordingly, the same elements are given the same referencesigns and will not be described repeatedly.

The electronic apparatus 1B displays images (still images, movingimages, live view images). The electronic apparatus 1B includes adisplay unit 50 and a second system control unit (control unit) 76. Thedisplay unit 50 of the electronic apparatus 1B has a configurationsimilar to that shown in FIG. 5. Accordingly, the same elements aregiven the same reference signs and will not be described repeatedly.

The first system control unit 75 includes a first communication unit75A. The second system control unit 76 includes a second communicationunit 76A. The first communication unit 75A and second communication unit76A transmit and receive signals to and from each other by wire orwirelessly. The first system control unit 75 includes, for example,elements equivalent to the division unit 71, image capture control unit72, and recording control unit 73 of the elements shown in FIG. 7. Thesecond system control unit 76 includes, for example, only an elementequivalent to the display control unit 74 of the elements shown in FIG.7.

The elements shown in FIG. 7 (division unit 71, recording control unit73, and display control unit 74) may be disposed in any of the firstsystem control unit 75 and second system control unit 76. Specifically,all the elements shown in FIG. 7 may be disposed in one of the firstsystem control unit 75 and second system control unit 76, or some of theelements shown in FIG. 7 may be disposed in the first system controlunit 75, and the other elements may be disposed in the second systemcontrol unit 76.

Examples of the image capture device 1A include digital cameras,smartphones, mobile phones, and personal computers which each have imagecapture and communication functions. Examples of the electronicapparatus 1B include smartphones, mobile phones, and portable personalcomputers which each have a communication function.

The first system control unit 75 shown in FIG. 17 is implemented when aCPU (not shown) performs processing on the basis of a control program.The second system control unit 76 shown in FIG. 17 is implemented when aCPU (not shown) performs processing on the basis of a control program.

In the configuration shown in FIG. 17, the image processing unit 30 andfirst system control unit 75 may be formed integrally with each other.In this case, the functions of the image processing unit 30 and firstsystem control unit 75 are implemented when a system control unitincluding one or more CPUs performs processing on the basis of a controlprogram.

While the present invention has been described using the embodiments,the technical scope of the invention is not limited to the scopedescribed in the embodiments. Various changes or modifications can bemade to the embodiments without departing from the spirit and scope ofthe invention. Further, one or more of the elements described in theembodiments may be omitted. Any forms resulting from such changes,modifications, or omission fall within the technical scope of theinvention. Elements of the embodiments or modifications thereof may becombined as appropriate and used.

The block arrangement patterns in FIGS. 8(A) to 8(C) are set such thatthe first and second regions have the same area. Specifically, theseblock arrangement patterns are set such that the first and secondregions have the same number of pixels. The block arrangement pattern inFIG. 8(D) is set such that the first to third regions have the samearea. Specifically, this block arrangement pattern is set such that thefirst to third regions have the same number of pixels. However, a blockarrangement pattern may be set such that respective regions havedifferent areas (the different numbers of pixels).

While the color filters 102 form a Bayer array in the embodiments, theymay form other types of arrays. Each unit group 131 only has to includeat least one pixel. Each block also only has to include at least onepixel. Accordingly, it is also possible to capture images on imagecapture conditions which vary among the pixels.

In the embodiments, part or all of the drive unit 21 may be included inthe image capture chip 113 or signal processing chip 111. Part of theimage processing unit 30 may also be included in the image capture chip113 or signal processing chip 111. Part of the system control unit 70may also be included in the image capture chip 113 or signal processingchip 111.

While both the gain and frame rate are changeable as image captureconditions in the first embodiment, at least one of these only has to bechangeable. While only the case in which the image capture conditionsare set in response to an operation of the user has been described, theimage capture conditions may be set automatically in accordance with themain subjects or the like.

While the block arrangement patterns of the embodiments are exemplifiedin FIGS. 8(A) to 8(D), other block arrangement patterns may be employed.While the case in which the sizes of the block regions are previouslyset has been described in the embodiments, the user may set the sizes ofthe block regions.

In the first embodiment, the division unit 71 recognizes subjects on thebasis of live view images and then sets regions. Alternatively, thedivision unit 71 may recognize subjects on the basis of an image withrespect to which the user has just pressed the release switch or movingimage switch halfway and then may set regions.

DESCRIPTION OF REFERENCE SIGNS

1,1B . . . electronic apparatus, 1A . . . image capture device, 20 . . .image capture unit, 30 . . . image processing unit, 31 . . . imagegeneration unit (moving image generation unit), 32A . . . detectionunit, 50 . . . display unit, 51 . . . display panel, 52 . . .touchscreen, 53 . . . second display unit, 54 . . . second touchscreen,70 . . . system control unit, 70A . . . first system control unit, 76 .. . second system control unit, 71 . . . division unit, 72 . . . imagecapture control unit, 73 . . . recording control unit, 74 . . . displaycontrol unit, 100 . . . image sensor

1. An electronic apparatus comprising: an image sensor includes multiplepixels arranged side by side in a row direction, a first control linethat is connected to a first pixel including a first photoelectricconvertor that converts light into an electric charge among the multiplepixels, and that outputs a first control signal for controlling thefirst pixel, a second control line that is connected to a second pixelincluding a second photoelectric convertor that converts light into anelectric charge among the multiple pixels, and that outputs a secondcontrol signal for controlling the second pixel; and, a display thatdisplays a first image generated based on a first signal output from thefirst pixel and a second image generated based on a second signal outputfrom the second pixel.